Vision target based assembly

ABSTRACT

Methods and apparatuses for assemblying, handling, and fabrication are disclosed in which targets are used on objects. The targets can be specifically applied to the object, or can be an otherwise normal feature of the object. Conveniently, the targets are removable from the object or covered by another object during an assemblying process. One or more robots and imaging devices for the targets are used. The robots can be used to handle or assemble a part, or a fixture may be used in conjunction with the robots. Conveniently, the CAD design system is used in designing the targets as well as for the assembly process using the targets. A plurality of targets can also be used to monitor and inspect a forming process such as on a sheet panel.

This application is a divisional of application Ser. No. 07/875,282,filed Apr. 29, 1992 which is a continuation of application Ser. No.07/478,078, filed Feb. 9, 1990, U.S. Pat. No. 5,148,591, which is acontinuation of application Ser. No. 07/110,541, filed Oct. 20, 1987,now abandoned, which is a continuation of application Ser. No. 061865,637, filed May 14, 1986, now abandoned, which is a continuation ofapplication Ser. No. 06/660,280, filed Oct. 12, 1984, now abandoned,which is a continuation-in-part of application Ser. No. 06/348,803,filed Feb. 16, 1982, now abandoned, and a continuation-in-part ofapplication Ser. No. 06/453,910, filed Dec. 28, 1982, now abandoned, anda continuation-in-part of application Ser. No. 06/651,325, filed Sep.17, 1984, U.S. Pat. No. 4,769,700, which is a continuation-in-part ofapplication Ser. No. 06/323,395, filed Nov. 20, 1981, U.S. Pat. No.4,482,960, and a continuation-in-part of application Ser. No.06/592,443, filed Mar. 22, 1984, U.S. Pat. No. 4,602,163, which is acontinuation-in-part of application Ser. No. 06/262,492, filed May 11,1981, U.S. Pat. No. 4,453,085.

BACKGROUND OF THE INVENTION

The following are co-pending applications in the same field showing thestate of the art which are herein incorporated by reference:

1. Method and Apparatus for Automatically Handling, Assembly or Workingon Objects, or "Targets", Ser. No. 348,803.

2. Robot Calibration, Ser. No. 453,910

3. Electro-Optical Systems for Control of Robots, Manipulator Arms andCoordinate Measuring Machines, or "Robots and Manipulator Arms", Ser.No. 592,443.

4. Robot Tractors Ser. No. 323,395, now U.S. Pat. No. 4,482,960.

5. Robot Tractors, Vehicles and Machinery, Ser. No. 651,325.

These applications and the present invention are all commonly owned.

Flexible robot assembly is often very difficult in the absence ofmachine vision sensors to guide the operation. Even with such sensors,operation must be both accurate, ultra reliable, and fast enough to bejustifiable relative to human labor. These criteria are seldom met bypresent day vision systems employing arbitrary gray level images and thelike.

The target based invention described in reference 1 above has wideapplication to the assembly process. Described therein are severalembodiments illustrating target based techniques which can overcome thelimitations of present systems. The key to the use of the disclosedsystems is that the target points on the part are easily discernable andunambiguous, after processing using rapid devices and other high speedanalysis software.

The target system functions well because it is based on high contrastimages and mathematical equations. To use targets one must know the partfeature data base relative to the target points on the part. Targets ontooling, pallets and fixed members may also be of use. Special retroreflective targets give best contrast, but targets can be holes, cornersor other easily determined natural part features.

Finally, where special targets are used which would not normally bepresent, techniques are disclosed to make these unobstrusive.

SUMMARY OF THE INVENTION

In accordance with the present invention, a method for controlling anassembly process is provided in which at least one first robot holds afirst part in relation to a second part. Targets are provided on atleast one of the robots or the first or second part. A TV camera thendetermines the position of the targets. From this determination, theassembly process is controlled.

In a preferred embodiment, the part held by the robot has a face onwhich the targets are located such that the TV camera can view thesetargets. In this manner, the robot is guided to handle or assemble thepart.

A method for fabrication or assembly in which a fixture is provided isalso disclosed. The location of a part having targets is determined witha TV camera and a robot then places the part on the fixture dependingupon this determined location. A further object may be located on thefixture.

During a dynamic fabrication, it is also possible with the presentinvention to target the part so that corrections can be made duringprocessing. Preferably, the targets are located in a finished assemblyin such a manner as to be out of view. These targets can be covered upby a second object assembled with respect to the first object ifapplicable.

The present invention also includes a method for guiding objects intoopenings or onto protrusions. A plurality of target features areinitially provided adjacent an opening or protrusion. These targetfeatures are imaged with a TV camera in order to determine the locationof the target features. From this determined target location, an objectis guided onto the opening or protrusion.

The present invention also makes use of a CAD system in a method offabrication. Initially, the design of a finished assembly of parts isprovided in a CAD system. The CAD system is also provided withindividual part designs which are used to make up the assembly. Thesedesigns include special targets or natural features which serve astargets. The entire system is then provided with a logical progressionof functions feeding program/functions to one or more robotic automationdevices for assembly. Assembly is assisted by the use of TV cameraswhich are provided with the design data. The parts are then suitablyassembled using images of the targets determined by the camera.Initially, the CAD system can be used to simulate the fabricationprocess.

In another preferred embodiment of the present invention, targets areprovided on an object before a forming operation is performed. Thelocation of the targets after forming are then determined. From thisdetermined location, the attitude, shape, or dimension of the object orportions thereof are also determined. Depending upon the determineddata, the handling, assemblying, inspecting, or working the object isthen effected.

Other features and advantages of the present invention are stated orapparent from a detailed description of presently preferred embodimentsof the invention found hereinbelow.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates a targeted assembly fixture with partloading by a robot.

FIG. 2 is a simulation block diagram.

FIGS. 3a and 3b schematically illustrate a dual robot version of theinvention, with specialized target concepts including targets on twosides of parts, on end effectors and the like.

FIGS. 4a and 4b schematically illustrate, respectively, an assemblytechnique using targets which surround an opening and targets which arecovered up by other parts or assembled to point the targets away fromview.

FIG. 5 schematically illustrates transformation of target data bases dueto forming.

FIG. 6 schematically illustrates flexible "fixtureless" roboticassembly.

FIG. 7 schematically illustrates an assembly cell according to theinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 illustrates an assembly operation using a robot 10 with a controlcomputer 11 to place a part 12 of a car on an assembly fixture 15 whichhas been targeted according to the invention. As is desirable, the CADsystem 30 (which has created the design for both the part and/or thefixture) has recorded the target data points 40, 41, 42 and 43 that areto be used by a camera unit 35 to locate the part 12 and/or the featuresof the part 12 that have been designated as targets plus the targetpoints of the assembly fixture. It is noted that in some cases assemblyfixtures are not required as pointed out below. When this data isobtained, the CAD system 30 downloads to the robot control 11 thedescription of the target point locations and the robot then homes in onthese points using the present invention.

In a typical example, a hood inner support is part 12. Part 12 is loadedin a programmable manner from the download data onto locators 17, 18, 19and 20 on a fixture 15 where targets 35, 37, 38 and 39 have been placed.Targets 35, 37, 38 and 39 are either on or near the locators 17, 18, 19and 20, or on points 50, 51, 52 and 53 around the framework of thefixture 15 which are related to the locators 17, 18, 19 and 20. Thecamera system 35 located on the robot (or alternatively off the robot asshown) locates these points and guides the robot 10 in. Differentialclosing data between part and locators can be provided to allow a smoothclose and mate operation.

To assemble hood outer panel 60 to the inner part 12, adhesive forexample, is first applied to hood inner part 12. The adhesive isconveniently applied using the same robot 10 and using the fixture orpart target data to guide the robot 10 (although once the part 12 iswell-located on the fixture 15, it is known accurately enough inposition to not require additional vision sensing except as averification check).

The robot 10 goes and gets the hood outer panel located in rack 61 usingeither of two methods. First, it can locate the hood by its corner edges70, 71, 72 and 73 as target points, grasp the outer panel 60 with asuction cup, and place it on the inner support part 12.

Another method is that the robot 10 grasps the panel 60 arbitrarily atthe best estimated location on its outer surface. The robot 12 thenpresents the rear of the surface of panel 60 having holes 80, 81, 82, 83to camera 35. Camera 35 then establishes the part reference to the robotso that the hood outer panel 60 can be reliably placed over the innerpart 12 and joined.

The fixture design and the target relationships to the part design canbe optimally matched in the design computer 100 as shown in FIG. 2. Thisallows a simulation routine 102 of the whole process at a point beforeconstruction. Unlike other hypothesized robot simulation activities,this one is based on the sure knowledge that the target points will beidentifiable within any given practical accuracy range because onlyhighly well defined targets are utilized which are at sufficientlyprecise locations to the data bases of the part or fixture to assuresuccess.

As shown in FIG. 2, simulation routine 02 begins with design part box104 which takes account of the design data of the part, including anyspecial points (if any). Next, from this data, target points aredesignated on the part as shown in box 106. The operational parametersare then specified as indicated by box 108. These parameters includecamera-processor types, type of robot(s), distances, range of possibleorientations, tracking algorithms used, photogrammetric algorithms used,special target indicators, and the like. Using the operationalparameters selected, an operation such as assembling or handling istried as shown by box 110. The results of this "try" are then determinedas indicated by box 112. These results are such things as cycle time,functionality, etc. Depending on the results obtained, variousparameters are varied in an effort to improve the process as shown bybox 114. Among the parameters which might be varied are the following:change equations, redesign part, change target points, apply specialtargets, add light sources, and add front end processor.

After the appropriate parameters are varied, the operation is againtried as indicated by box 116. If not successful, the parameters areagain varied and the operation rerun until a successful operation isachieved. At this point, the operation is ready to be integrated withother operations as indicated by box 118. It should be noted that CADsystem 100 would have ideally programmed the parameter (speed, accuracy,risk etc.) of various target choices to be seen using various cameralighting, image processing, and the like.

In addition, simulation 102 can continually optimize the solutions ofthe photogrammetric target equations to allow the best performanceaccuracies to be obtained within the desired cycle time of theoperation. For example, the target choice and location, number oftargets etc., can be changed on a CAD system and operation of theprocess using robots interacting with these targets simulated andphotogrammetric and/or tracking equations optimized.

A simulation block diagram is shown in FIG. 2. With the system, itbecomes cost effective to develop simulation programs for robot systemssince one has an assurety of vision based guidance. This is not truewith other guidance systems in operation today, and particularly not sowith those capable of the high speed accurate operation required tojustify cost in modern manufacturing.

FIG. 3a illustrates a variation on this particular embodiment in which adual set of robots is used. In this case, Robot A picks up the part 300and presents it to Robot B which then inserts a shaft part 301 into asleeve 302 in part 300. This assembly operation could be any other typeof operation as well.

Part 300 is picked up by the robot gripper 304, preferably, but notnecessarily, using target points 310, 311, and 312 on part 300 viewed bycamera 321. When it is presented to part 301, the second robot caninteract in any one of several ways. In the first case, the part 300 hastarget points that can be identified such as holes or other purposelyplaced targets that can be identified relative to it. In this case, thecamera unit 320 on Robot B can home in on those points. Again, asbefore, all of this data can be downloaded from a central computer 350.

Another possibility is to target the Robot A end effector or tooling 304holding the part 300, as shown with targets 340 and 341. Since part 300has been picked up originally in an accurate manner relative to thetooling 304, the tooling 304 then can act as the data base to providethe target data points for the second Robot B. This is particularly truesince these target data points can be idealized targets comprising retroreflective material or active fiber illumination which are almostperfect in the ability to be accurately judged. Thus, minimal stackuperror due to this transfer of location data occurs.

It is noted that the camera 321 on Robot A can "home-in" on targetedtooling plate 360 on Robot B having 3 targets as shown while the cameraon Robot B can look at tooling 304 on robot A. Each one can thus home inon the other.

This ability of Robots A and B to look at each other is illustrated inFIG. 3b. A targeted plate 361 surrounding camera 360 is attached torobot arm 362, while a similar targeted plate 371 is adjacent camera 370on robot arm 372.

Note that robot arm 372 could, for example, be instead a fixed structuresuch as a drill or router unit with camera 360 (or 370) homing the robotarm 362 in on it with a part in its gripper using targeted plate 371 (or361).

A third possibility is to target both robots in a manner visible fromabove (or elsewhere) and use an overhead camera system 390 to monitorthe closing target conditions of one to the other. This is schematicallyshown in FIG. 3a.

Another version is a front/back target situation where control transferis made between target sets on a part. A part can be picked up (say byrobot A) based on targets on one side, and presented to a camera onrobot B whose camera locks in on targets on the other side.

FIG. 4a illustrates a robotic arrangement for putting parts into holesor other openings. The area around the hole 500 in part 499 is targetedwith targets 501, 502, 503 and 504. Typically 3 or more targets areutilized to allow the total 6 degree of freedom approach to the holes tobe made. In this example, the camera 510 on robot 530 senses the targetpoints 501, 502, 503 and 504, and knowing their locational relationshipsto the hole 500 and its centerline, guides the part 520 such as a pininto hole 500.

TV camera 510 can be located on the robot arm, or elsewhere. The camerasensor unit may also be augmented by triangulation sensors using a diodelaser or other light source 540 to project the laser spot 541 or otherzones onto the part to further aid the guidance. Such spots or zones canalso be viewed by the same camera 510.

The targets around the hole can be anything suitable. For example, on acast iron block or a cast aluminum block, these can be cast inindentations, raised areas, etc. perhaps with special signatures (e.g.triangles, stars, etc.) to allow their easy acquisition. A variety ofother possibilities are also in order. The lighting for such targets isusually provided by a source on the robot arm although the maximumcontrast, indented, or raised targets should be sometimes lit at anangle (see copending application reference 1 on lighting for robotapplications.)

The particular opening can be any opening, not just a hole. For example,the points at which a gas tank lid is to be placed onto the side of acar rear quarter panel could be used. In this case, the target pointsmight be painted onto the body side outer surface, or other innerpoints, which would be covered up by the lid.

In this general case, the target points on the surface are covered upafter the part is assembled on the shaft by the part. This way they areremoved from aesthetic considerations. There are numerous examples wherethe object involved can be used to cover up the target points that werefirst used to find the 6 degree of freedom or multi degree of freedomlocation of the object.

For example, consider FIG. 4a where the part could be a radio knob witha disc 598 (dotted lines) on a shaft 520. When assembled into hole 501in dashboard 499, disc 598 covers up targets 500-504.

It is noted that hole 500 in FIG. 4a could also be a protrusion like ashaft, stud, etc., assembly to which also requires multi degree offreedom guidance.

Target points can be located on the end of protruding objects as well.For example, it is often necessary to insert something with multipleholes onto multiple pins (or vice versa). This is the case of wheelbolts on a car. In this case, illustrated in FIG. 4b, the targets arethe ends of the wheel bolts 601, 602, 603 and 604 which are suitablytargeted according to the invention, for example, with reflective paint.Conveniently, the camera 620 can be mounted in the tooling 621 holdingthe wheel 622 so as to look through the wheel at the bolt ends.

Another example of targets which can be camouflaged is the use oftargets on the back side of an object as was illustrated in FIG. 1. Inthis case, it is desired for example to assemble a part which has beentargeted for ease of manipulation to another part. The targets in thiscase are purposely chosen to be on the inside when the assembly iscomplete.

For example, let us consider a process where a robot utilized forassembling a house is used. The robot picks up targeted board lumberwhich is targeted on its back face with the target points, or what willbe the back face in the home. Th back face is then presented by thisfirst robot to a second robot which nails the board to other boards thathave already been assembled. The assembly is done in such a manner thatthe target points that have been used are mounted on the inside. Indeed,this works out quite well since the robot can also look at target pointson the outside of the rest of the structure and determine from them thelocation to place this first part over same. As the structure is totallyassembled, all targets end up pointing inside (or are covered up). Thisallows the use of such targets without creating aesthetic problems onthe outside.

This coverup was shown relative to the radio knob in FIG. 4a but alsowould be the case here on the inside where one would cover it up laterwith wallboard.

Consider now FIG. 5 which illustrates the use of target points 700 allover a structure 702 and particularly on formed parts. The intent hereis twofold. The first is to deduce from the changes in the targetpattern the effects of forming on the part 702. For example, it may bedesired to obtain the new dimensional shape of the part 702 using thedata base now resulting from the forming process 706 and any strains orany in-plane deflections which may have occured as a result of theforming or welding process, etc. This can be used to control the process706 as well as feed data forward as to the new shape for further roboticassembly.

For example, consider the case of metal body panels or aircraft panels.They start out as steel blanks 702 which can be imprinted with a dottarget pattern on one inch centers throughout the total width. Thetargets 700 can be rather precisely applied, or if not, can be initiallymonitored by a camera means 708 to discern what the pattern is. Suchapplications could be through a roll coat 704, an etch process, or thelike.

This sheet of steel 702 is then formed into a new form such as a cardoor or, in a more simpler case, a deck lid. It is now desirable to usethose target points 700 as viewed by camera means 710 which have nowchanged their form to determine any irregularities of the formingprocess as well as to establish the new data base for the part. Wherethis data base (namely shape and so forth) is a known desired one as itis in the case of a door, one would then like to compare the desireddata with that resulting from the new pattern. Furthermore, this newdata base can be utilized in the actual handling of the part betweenstations of a press line, for example, as long as it is known what toanticipate in terms of what it is. Indeed, it is noted that rather thanputting down a roll coated impression every one inch at squares on theinitially formed blank, one can actually roll coat down a series of dotsof other target points which make a square pattern when optimallyformed.

Now, take this case one step further, where a formed part 702 such as adeck lid has dot points 700 on it. This formed part is then to have apart 712 welded to its rear by spot weld robot 716, which part 712 isthe inner stiffner of the deck lid. The dots 700 are viewed by a camerameans 714 and not only used in the handling process of handler 718 tomove this part 712 into place also using camera means 714, but also toposition the welded inner part 12 relative to part 702. These dots arefurthermore used to check after welding whether or not certaindistortions have occured beyond limits in the welding process and toestablish the new data base for the part.

If new data base is satisfactory, the part is moved on by a suitablecamera equipped handling robot to shipping or to the body build process.In this process, this formed part is mated up with other parts and againthe target points 700 can be used in this mating, for example, inputting the deck lid into a deck lid opening.

In the case of doors, it is particularly important because the door isloaded with hardware and then hung from hinges which can tend to furtherdistort the door. Indeed, if all the formed parts were made up andtargeted in such a manner, it would make the job much easier for theautomation and inspection through the total build of the part. Thisconcept goes well beyond formed parts, to all types of things. However,formed parts are the ones in which the most distortion can occur.

In the final operations, the etched on or painted inked on marks, etc.,if on the outside of the car, are generally painted over or otherwiseremoved. Naturally when such marks occur on the inner portions of themetal, no aesthetic problem exists.

This process is particularly applicable to formed metal which can springback and generally has a relatively unknown shape such that errors intarget point locations are not severe relative to the accuracy of themetal.

Another process that is somewhat similar is casting which is arelatively loose process on the same tolerance level as the forming.Plastic molding is another with the same tolerance but unfortunatelythis generally does not lead to distortion except during perhaps glueingoperations. In any of these cases, suitable target points can be moldedin, cast in, etched on, burned on as with lasers, or any suitable meanscan be utilized.

One can also use an absolute encoded target pattern, one which has noambiguity as to where each target is. This could be with clusters oftargets on different centers or patterns, different shaped targets, orthe like.

It has been noted that this process is most applicable where moreoperations occur to a given part that can change its shape and where thepart is more likely to change shape as a result of the operation (as inspringback). This means that the roll forming, hydroforming, pressforming, press brake forming and other forming operations done in theautomobile, aircraft, and offroad vehicle industry, particularly arenatural users of such technology. By judicious choice of targets andtheir applications, one can utilize these targets throughout the processfor material handling assembly and inspection purposes and generallyeffect large cost reductions in the manufacturing as well as improvegreatly the flexibility with which such manufacturing can occur.

The target application techniques can be whatever is suitable. As isobvious, the closer to a flat blank that the targets can be applied in,the better from the purpose of target application as long as thosetargets through the forming process can maintain their legibility.

Speaking of legibility, the typical technique is to utilize a solidstate TV camera to look at the target points. This has provedsatisfactory, for example, to look at etched lines on die blanks inpress forming.

While target points on the formed materials and other shapes areconsidered here mainly in the light of formed steel stampings, aluminumgraphite epoxy and other formed parts, one can actually extrapolate thisto all types of structures and parts such as, for example: plywood onbuildings, concrete which sets and takes a shape and then might later beused for further operations, ground (earth) as in targeting points onthe ground which would then be used later on to guide particular itemseven though the ground had shifted or was distorted by working aroundit, and the like. Indeed, it is the general case that if one can gettargets onto the surface of structural parts or other objects in theiroriginal form before any further work and if those targets stay on thoseparts throughout the work, then they can be used the maximum number oftimes and therefore increase the cost justification of the targetingconcept. For targeting on concrete, targets such as glass beads, dottargets, or cats eye retro reflectors could be actually inserted intothe material where they would harden therein and still be visible fromthe outside. These, of course, are idealized targets. Others wouldsimply be little dots, points, or even non-circular targets. In fact,non circular ones could be advantageous from the point of view ofabsolute target coding. However, in general, circular targets aredesirable.

One must consider the integration of all these target concepts. Forexample, discussed above is targeting the points in their bare form andlooking at a changing data base for the part as it goes through thevarious forming and joining operations. Also discussed was designing thepart in the computer with the targets present thereon, the targets beingin this case either special targets or normally appearing portions ofthe part or both on any one part or on any grouping of parts.

In other copending applications such as those referenced above in theBackground section, other systems for targeting the work area of robotsand automation of guiding vehicles and other target functions areperformed. The operation too of the various vehicles and robots can besimulated in the computer relative to these target points as well.Again, a key item is that with target points one is much more assured ofaccurate reliable operation and therefore the justification for suchsimulation and expense thereof is paid back by the surety of thefunction of the final simulated structure. And too, one is dealing withmathematical representations of target points and photogrammetricequations, not relatively unknown functions of vision systems with graylevel images of object features. Simulation of dynamic target trackingalso become feasible as the problem is much more defined than with graylevel scene analysis. This also leads to faster assembly and morejustification.

One can indeed state that the whole work area and work environment ofautomation in the factory of the future could well be targeted. All theparts, the fixtures, the jigs, the tooling, the robot worked areas, thepassageways and areas of travel of automated guided vehicles carryingthe parts, the transfer conveyers, even belts on which the parts ridecould be targeted. With such target data stored in the data base of amaster control computer for any one cell or an overall host computer,one has complete control of the environment. Everything is knownrelative to known target points such as: the location of the automation,part, the part shape, material handling, bringing the part and taking itaway, relationship of other parts to the part, and the like. The datarelative to this information can be manipulated in the computer andoptimum operations derived such as trajectories of robots, forming, andthe like. It is considered within the purview of this invention tointegrate these functions to the maximum extent possible to whereflexibility s attained while still having complete control over theoperation.

FIG. 6 illustrates robotic fixtureless assembly using targets andfurther illustrates the concept of dynamic target location to adjustholding or welding robots dynamically under target control in responseto forces (clamping, welding) tending to change the part from itscorrect shape.

As shown, car cowl 802 is to be welded to flange 803 of underbody 801,under control of at least one target sensing camera 800. Camera 800senses various targets 804 on cowl 802 and targets 809 on underbody 801itself. The sensed data is then fed to computer 808 which is connectedto holding robot 805 and welding robot 810.

Robot 805 has gripper 806 holding cowl 802 in place and having targetplate 807 observable by camera 800. Robot 810 has spot weld gun 851putting flange to cowl welds on, with target plate 850 also observableby camera 800.

The holding robot 805 is moved into position to the correct locationdetermined by cowl 802 and underbody target location and the weldcommenced. If distortions occur moving any part to a wrong position, theholding robot 805 and welding robot 810 acting in concert under controlof computer 808 move to correct them. Typically, a plurality of weldingand holding robots would be utilized. Some non-adjustable ornon-programmable clamping or welding could also be used in conjunction.

The target's data base provides reliable accurate control over thisoperation and allows instant feedback correction. The procedure isflexible and could be programmed for any operation or car line.Typically, several cameras looking from different views would berequired on a complex structure.

Note that when such a fabrication cell or an assembly cell is fixed allin one spot (no moving conveyors), the camera computer 808 can memorizethe position of each of the pieces in build up, from its targets, eventhough they are later covered or partially covered up. For example, whencowl 802 is positioned to underbody 801, computer 808 records theposition of underbody 801. Then, after cowl 802 has been welded tounderbody 801, computer 808 again records its position. As a next step,computer 808 could guide A-pillars in each side from their targetsrelative to the recorded cowl position (and if desired, consideringunderbody 801, too).

For example, one might purposely slightly change the A-pillar locationsrelative to the cowl if another body portion mating to the A-pillar wasrelated more to underbody position. This avoids stackup of errors. Thisreally is a major advance in that it accounts for weld distortion--youcan measure and correct while welding and after weld you know the finallocation.

Correction of distortion or stack up of tolerances in real time isabsolutely a major advance. This would apply to adhesive bonding,riveting and other joining processes as well.

The term "TV Camera" in the invention generally connotes a dimensionallystable type, such as a solid state TV camera, e.g. a GE TN2500. However,other means for electronic imaging can be used such as Vidicons, imagedisectors, linear arrays and the like. Indeed, a mechanically scannedlaser beam, like a flying spot scanner, can be used to sweep throughspace in order to find target points some of which envisioned could beparticularly reflective or responsive to laser light. The primary goalis to find target points or features. However, the TV cameras also allownormal gray level part images to be obtained, a big seconday advantagesince this is often combined with the target based guidance function forinspection, recognition or other purposes.

A suitable target sensing vision system is that described in the priorcopending applications noted above, and it is noted that the designprogram on the computer aided design system should be able to design thetype of location of these targets which are suitable for the visionsystem.

The laying in of the targets could be by specifying operations which putthe targets on as special items, i.e. by spraying them on, sticking themon, or drilling special holes or the like. The other idea is to use theCAD system to design the part, and then to look at the part as designedin its functional way. Drawing upon the knowledge of what can constitutea natural target (as in a naturally occuring hole or corner), suitabletargets are specified. A computer program can be used, if desired, toaugment the decision as to which natural targets are the targets to beused.

The beauty of the computer design system then would be to allow you to"try out" the design with a hypothetical target set using a typicalsimulation program for a RPS (or other target determining camera system)equipped robot.

Suppose one would wish to design an operation where a sheet metal doorinner which is full of holes is to be attached to an outer panel bywelding. One robot holds the outer, one holds the inner. The robotgrabbing for the inner would use a camera system located on its endeffector (or near its end effector) and guide the robot in using certainholes on the inner panel as the targets. The CAD system program would bedesigned in such a way as to allow a three dimensional simulation of therobot's movement to be made, and to determine the robot cycle time whichwould be achieveable in the accuracies of position using those targets.This would be done by working through the known accuracies of the camerasystem, and the photogrammetric equations could also be utilized (whichwould be called upon from memory) and the whole system designed on theCAD system. The other robot picking up the door outer would work in asimilar way, and other points on the door outer that could be picked upwould be monitored and the total assembly predicted using the controlequations of the two robots.

Each operation involving a particular part that might go through a linecould then be tracked. For example, now that the inner and outer trackare assembled, the assembly could also be sensed from the holes on theinner or other features in the operation of putting the door onto thecar. This too could be automatically tried out on the computer.

The key here is that the target type solution allows us to quantifyeverything. Every single possibility can be predicted by mathematicalequations which can be put into a simulation program used to try outdesigns.

One key advantage obviously is that if the whole operation is not fastenough or accurate enough, other types of targets might be specifiedthan certain naturally occuring ones. One would then know that inplanning the line layout for this operation, and one might have to usetwo robots per operation instead of one, etc. All of this could be knownin advance.

Also, the CAD system could design the way in which the differentstations interact on a line. For example, certain targets might becovered up by assembly operations and alternate targets would have to beused. Indeed, complete alternate strategies in case certain targets weremissing could also be programmed into the CAD system and the wholedesign checked out so that such targets were present in the initial CADdesign part.

The assembly operations usually require the build up of components ontargeted tooling which is used to hold the portions in place while othercomponents are added. If such tooling is targeted, the location andprecision of the robots, etc. doing the assembly can be vastly reduced.

We should consider the problem of assembly with and without toolingplates. In one case, where the robot holds the part to be assembled, theother robot assembles another part to it having a target. Perhaps eventhe first robot's gripper is targeted, which in essence then is aprogrammable tooling plate. In fact, changeable grippers can be used, ascan changeable tooling plates, all with targets to facilitate theirchanging.

Continuing the ideas of these RPS things, it is not just usable ontooling fixtures for assembly. It is also welding or any other kind offixtured operation where one would put the object onto the tooling andthen add another object to it. Any sort of holding fixture will do.Again, another robot (if it is still enough to act as the fixture on theother side) can be used assuming the part is targeted.

Also covered in this case is the concept of covering up targets on anobject when the object is assembled to it. For example, a radio knobattached to a targeted radio front plate with targets around the knobpost would be suitable. Also suitable is the assembly of a tail lightbezel to a car with targets around the attachment points.

For example, if everything could be put together from the targets in astatic location, this would be most advantageous. Also desirable is toonly move the part or whatever simply to 2 or 3 such static locations.For example, right now instrument panels are put together on acontinuous line where they go around a continuous loop with maybe 16,20, or 25 people standing in the loop assembling all clusters etc. intothe instrument panel.

Assuming that the problem of the wiring could be solved in an instrumentpanel line, one might replace this with a single fixed panel. FIG. 7shows a fixed assembly cell 900 for an instrument panel, engine, orother component 902 into which would be placed all of the various itemsby one, two, or more robots 904 going out and grabbing them fromdifferent containers 916 located nearby. There are certain advantages tocell 900, namely that one does not have to deal with a moving line andthat everything is fixed (which means the highest placement accuracythat could be expected.) In this condition, one would have to be able topick out all the different parts. There might be 20 to 25 parts requiredfor the panel, plus the multiplicity of different versions of the samething, which might lead to a total of 100 different choices with therobot. This would require either very well positioned parts or a goodvision system, and the targets would make it quite easy to do both fastand accurately.

Speed and accuracy would be paramount because if this system could notgo fast, it cannot possibly compete with humans on a moving line.

Most of the parts of the instrument panel or engine 902 as shown in FIG.7 could have targets on them, which could be made to seem like theexisting parts of the panel or engine 902 (which have lettering on themor certain points or corners and holes). These are all already presentin abundance on a panel or engine 902, for example. In addition, knobson the panel can be used to cover up targets and other whole assembliescan cover up the targets on the inside of the panel which would be usedto guide it in.

The central problem with this then may not be the targets at all but thetooling. If a universal robot is utilized, the robot might have to havedifferent end effector tooling and thereby lose cycle time changing it.Such a robotic work cell is thus very appealing. However, parts wouldhave to be brought in when they ran out from outside the periphery ofthe cell. In other words, what is essentially parts trains 906 wouldcome in on tracks 908 or 914, just like tracks into a roundhouse.

With a car body, clearly the body in the first place is made up of alarge number of panels welded together. This is done in-line, and ateach station a varying piece is brought in, fixtured and the car bodywelded up. This could conceivably be done without the fixtures usingtargets.

For example, suppose we wish to build a car body up from scratch. Wecould take stampings that comprise the side of the car, e.g., startfirst with the cowl and the A-pillar and assume that the cowl is weldedto some sub-frame. The A-pillar could be brought in using targets on thecowl and targets on the A-pillar. The A-pillar would be accuratelypicked up and placed by the robot on the cowl where it would then bewelded once it was in place. The accuracy of the positioning would betaken from the targets on the car, not from a fixture.

Satisfactory target accuracy could be achieved with an A-pillar roughly2 ft. long welded to a body cowl. This piece should be accuratelypositioned within +/-10 thousandths. Whether it is, in practice, isdebatable however. First, one would figure out just what the targeterror is to do this. In any case, with one robot holding the part perthe camera unit (which would now be not on the robot at all but off tothe side so that it can get the true reference of the A-pillar to thecowl), another robot comes in and welds it. This can be done withdynamic correction, if necessary, by the first robot for position due todistortion by the welding robot. This could be called "fabricationmethod with dynamic real time target location".

After putting on the A-pillar, one could then go over and put on theother A-pillar, so that one could come in the body side. Indeed, all ofthe parts could be welded on this way, particularly those internal ones.This is really just like a completely flexible fixture. With differentprograms foe different parts, one could make anything.

Using targeted outer parts, you can then drill all of the holes neededfrom the targets etc. Again, one would have to bring all the parts toit, so that it would then move the finished subassembly out of this celland bring another one in. Instead of having a continuous line, you wouldhave a relatively few number of stations, relatively widely separated.This would allow, however, different things to be made more easilybecause on a continuous line, the varying part steps would not match up(although the same technology could apply to that too).

To do an assembly like this, you might have multiple cameras too, oneabove the whole cell 900 and various ones off to the side coming in fromevery angle. This would allow you to get a better fix on each of theparts as well as deal with the fact that sometimes you would obscureyourself with the tooling doing the operation.

An interesting thing is a co-target, that is a target that tells wherethe other targets are. Such a co-target identifies the part and as suchtells where the targets are. It also gives the orientation coordinatesso that one can find the other targets.

The beauty of this one market idea is that the single market identifiesthe part and therefore all the targets around it. It could also give anapproximate orientation code for the part, which now could include notjust a reticle (which was shown in the turbine case noted above) butactual dot code or something from which an accurate orientation in 6degrees could be created. From that, one can then find the otherfeatures of the part that one wishes to use for further use such as thecorners, the holes, etc. Therefore, there is a one-step process usingthat single marker to tell the rest, including where the others arewhich in turn are then used.

Other kinds of targets include: a serial plate, either just as it is,specially designed, or with print markings; a logo; a bolt head (asmentioned above); and an instruction plate or a bar code marker.

Getting targets on sheet metal can be a problem as lots of sheet metalreally does not have any holes in it per s, at least on certain surfacesthat one might want to use. In such a case, we can paint targets onthose for the assembly operation noted. Marks such as smallindentations, ridges, tabs or the like can also be stamped in.

Painting could be done in a stamping fixture coming right out of thepress, since the part is trapped anyway usually upside down. It wouldnot take too much to lay it in another fixture and determine targetlocations. Another possibility would be to actually stamp the target in.This would seem to be impossible on an outer surface unless one stampedon as part of the stamping process something that would essentially comeoff.

The other obvious place would be in a weld fixture where one clearlywelds most panels together with some other panel. Obviously, you cannotreplace the weld fixture with a robot and targets if one does this, butat least the assembly of that welded up component to any other could betarget automated. Weld fixtures typically also have sensing associatedwith them to create an in-line check fixture situation. Even this checkfixture could actually contain the marking units.

Obviously there are numerous underbody and chassis components that couldbe targeted without any detrimental aesthetic effects--mufflers, tailpipes, brackets, clamps, tie rods, steering parts, etc. These areobviously labor prone things to assemble and one could envision robotsworking underneath the car while the other work is going on above in theassembly function. This is, of course, very tiring work for people andtherefore would be well set up to robots. There is no reason why thewhole operation could not be up above the floor. The car is brought inoverhead with some robots on a second tier and other ones underneath.

This system should be usable in a variety of circumstances, such as:components that are all assembled to the body in the trim area includingtail lights, glass, front, rear, wheels, headlights, grill, doorhandles, rear view mirrors, antennas; internal assembly components suchas door assembly, lift mechanisms, trim, and instrument assembly coveredabove; and underhood assembly (this too probably can be marked up in anyway desired with no detrement).

Note that another type of marking, purely decorative, is trim decalswhich could be judiciously chosen as targets.

So far all target sensing development is based on a raster scan camera.These, therefore, were generating windows, tracking, etc. Other types ofspecial cameras might be interesting for this application. For example,if one always has 4 targets you could have literally 4 image chips with4 separate scans of the field which would then simplify the processing.In addition, one really does not need all the number of lines that thereare in a field if one uses targets that occupy large portions of thefield. Therefore, a camera having 10 vertical lines and 10 horizontallines widely spaced might be quite suitable. Instead of scanning say 300lines with a camera, you would then scan only 20, but this wouldpresumably be sufficient. This would require a target design that wasable to be cut at multiple points and give the right answer. Naturally,the speed of response of this would be quite a bit higher.

A polar scan swept like a radar screen is also possible. Although theseare not widely available, one could assume some future value.

One of the beauties of putting targets on the A-pillar, etc. is that thefixture can actually slip, and dynamic correction be made via the highspeed of response to the target position.

One would also expect that you would have to have 3 or 4 targets to goalong with all of this or else you could not do it. The other thing isthat you would certainly want to have redundant targets in case some ofthem were knocked off or the paint gun did not work, or whatever.

One can also come up with a plan for automating the way in which theparts are brought in. Targeted little trucks would be provided thatinstead of having fixed tracks can actually reposition themselves atdifferent locations, either in the horizontal plane or even vertically,to be grabbed by the robot. This ability to have them at multipledifferent planes with a very high reaching robot and targets-also allowsfloor space to be conserved. Indeed, another robot could be used to loadthe parts into the racks from the targets out of the dunnage. In thatcase, therefore, the parts would be presented to the assembly robot in abetter orientation while still targeted, possibly saving some assemblytime. The other unload robot then would be used to essentially deal withthe messy conditions.

From the above, it appears that all things can be made from targets. Ifrobots can accurately position and are structurally stiff enough towithstand the forces of welding, drilling, or whatever (and this is notquite sure), then presumably all things can be made this way. However,some things apparently cannot. For example, if one has to back up adrill, this can be done with a robot in some cases, but not in othercases.

In a structure such as this, important considerations would be: wherethe cameras would be located, what kind of robots would be utilized,would it be better to have more than one robot holding each part orclusters of parts, etc. Particularly, the concentration should be on thetarget orientations and the cameras or the like.

As noted above, the following are parts of the car in which, the theory,targets would mean very little in terms of their add-on aesthetic value:under hood, under carriage, when covered up by something else as inseats over bottom part, inside door covered by trim, inside of taillight covered by light, grill covering other part, etc., inside trunkonly to a degree. Those are the main areas, but they may constitute atleast half of all the assembly work, if not considerably more (in fact,much more).

The other thing to be considered is can the body structure be built upwith targets and then the targets taken off. This seems logical. Everyweld fixture of every panel can be a place for target application.

Incidentally, there are people talking about painting the whole carafter it is assembled, not just sticking painted doors on. That would ofcourse make the job even easier. Then everything is targeted going inand you just paint over them.

If parts are targeted, then obviously, an RPS equipped robot can pick upa part in a very well known way. Assuming that the gripper is correctfor this, it can then (within the accuracy of the robot itself, whichalso could be RPS controlled) present this par to another robot in awell positioned manner. Ostensibly, the accuracy of this position couldbe just as accurate as if it would be fixtured, assuming that thefixturing details and the RPS were equal in accuracy. The only problemwould be the inherent inaccuracy of the robot.

However, it should be noted there is nothing to keep the second robotused to assemble something onto it from using targets as well, thereforetaking out that positional error. Conversely, the targets utilized bythe second robot could be on the first robot as on its gripper. In otherwords, if the first robot grabs the part in a known way, the secondrobot only needs to home in on the first one, not on the part. Thiscould make it a lot simpler for an assembly operation. A very clear-cuttarget board could be utilized on the gripper or somewhere on the end ofarm toolings that would be directly relatable in a known way.

This two robot assembly is very similar to the human using his twohands, one to hold the first part and one to hold the second, and indeeda twin armed robot could be considered for the job as well. It is morearms and hands than it is robots per se, although they have to beseparately controllable.

It is however interesting to note that the inherent accuracy of theposition is considerably higher than for a human. The human, if thereare no marks on part at all, would have a great deal of difficulty forexample, drilling three precise holes exactly positioned in a location.The human himself would need some kind of fixture as in a template orsomething. Wherein in this case, the targets actually provide that sincethey are carried on the part. They can be on there in a way that a humanwould not be able to directly relate to in an easy manner. Theadvantages over humans are noted.

There is a special assembly technique that lends itself to this. Oneclear-cut aspect is targeting parts in unobstrusive zones, as on theback side away from the observer. For example, a radio knob on which theback of the knob had the targets or a door handle of a car in which theinner door panel was targeted (this is almost a natural case anyhowsince it is full of holes which can be used as targets). If we then useone robot to grab at the inner side where the targets are present, wecan then assemble the outer side which might be a visible one ontosomething.

The outer is usually the part you would like to grip so that you couldassemble it to something else like the shaft on a knob or the body on acar. This is not easy to do if the targets are on the inner. What ismore straightforward, therefore, is to use the targets on an item thatit is to be assembled onto as in around the shaft or around where thedoor goes (the opening).

One possible way to do this is to have `U` shaped tooling where it isgrabbed on the rear part but held on the front via the `U`.

One thing is sure, you could take doors that way and use drilled trimthings on the outside doing it that way. This is if it all lined up. Wemight have to do this on the finished car, however, targeting theoutside.

One could also grab the part from its targets and then move it over to adrill fixture rather than a second robot. The fixed drill fixture orwhatever it is could be different for different cars say--this would ofcourse give it flexibility. The robot would take the part and move it toone of five stations depending on which type it was sensed to be, andthen orient itself relative to that station which itself would betargeted.

In the latter example, it is a gang drilling, gang welding or some othermulti-position gang operation with multiple points. This would be trueof a robot grabbing parts up willy nilly and Just putting them in theright thing. Note however, we are talking about non-fixtured parts inthis case. Parts that are not being dropped into a fixture, onlypresented to a gang drill that itself does not have any fixture locatorsbecause the targets have located it.

However, this might not work since in some operations the robot couldnot hold steady enough and you would have to drop it onto locators. Butin other cases, it would just present it to it. The interesting thingabout a robot, like a human, is that it could just go to one drill andsequentially drill four holes in a row using the targets to set up thestraight line or whatever the other pattern was. This could be totallyvarient depending on what part it was, without the necessity of buildingthe special gang drilling fixtures. This could be the putting ofmoldings on, but really it could be anything (even cylinder heads orsomething else) if one could position accurately enough and hold it, ofcourse.

One obvious thing that you do not have to position ultra accurately isspot weld guns. This would be relatively simple. You could sequentially,using targets, put a part set to a spot welding gun that would then putdifferent spot welds depending on what it was (or glue or anything elselike that). Therefore, one robot which would target on the gun that itcould aim the part off of, could actually do numerous parts in manyways, all programmably. However, such a robot would obviously not bevery fast as it is a one at a time deal. This would be the oppositeeffect of having gangs of spot weld guns on fixed automation coming inon a part that is transferred through. In this case, a single robotwould go make all the welds on a part, and it would be transferred out.It would take many more robots, but that might not be more expensive.You get to optimize the one gun you do have, or any other things such asadhesive sealer or guns or whatever.

The same logic holds for presenting a part that has numerous threadedholes in it to an automatic tapping machine or an automatic nut runneror for that matter taking a nut runner and moving it to different pointson a part.

One interesting question is whether all this allows whole other assemblyprocedures. If you do not need special tooling and fixtures, you couldjust put a car together by feeding parts to the assembly point such ascell 900 depicted in FIG. 7 just as if a human was going to stand thereand build a car in his garage. He would go out and get the parts andbring them in and assemble them up. The advantage of this is that youare moving the parts by conveyors to the build points, rather than thecars to separate build points. The robots 904 are going out getting theparts and coming back. Robots, for example, could be mounted on slides910 or for that matter could be gantries themselves. This would looklike a round house or something with each of the slides then being fed.However, you still have to get the parts in and out of the system or thefinished cars have to be moved as by conveyors 912 and 918.

It does mean that the car as it is built up is fixed in a known place.Thus, each place you do something is dimensionally known from theoperation before. This is not true in line type assembly unlesselaborate shot pining is taking palce. Therefore, the total build up ofthe car is a known commodity even if the positions of the parts and theother things feeding into it are unknown.

In this new mode, each of the parts of the car is targeted and as theyare laid upon each other the assembly itself becomes targeted insofar asthey are not covering up the old pieces. This means that as each robotapproaches it can memorize the locations or use the targets. Indeed, thetargets on some of the parts as they are positioned offer a chance toupdate the robot coordinate system so that it can re-memorize thepositions in case the robot gets out of sync.

For welding, a robot would bring the parts in from one side, lay themdown and the second robot would grab arc welding tooling and weld themall. Such a method that could be totally programmed in a CAD system.This completely relieves having to transfer data to fixture builders,etc., shaking down lines and so on since one can theoretically do thisin CAD and you are dealing with minimum numbers of pieces of hardwareand delivery time therefore. One robot, perhaps two robots would beworking on a single area, each with different sets of tooling that theycan use. Different robots might pick up parts from different sides withan overhead gantry type robot as well.

This whole procedure would just keep proceeding until the car was built.Obviously there cannot be much room for parts washers in this type ofthing. Instead, the car would be transferred out on conveyors 912 and918 after the body was welded up to a second location where it would bewashed and then transferred in to another similar type station forassembly. In this case, all parts would be brought in from the outsideand the assembly built up. One might do this in two stages, assemblingthe inner parts of the car and then using the assembly robots to goahead and weld the top on. This type of procedure allows one tocompletely change ones mind as to how you are making the car. Make itone way one day and another way the next. You do not have an investmentin the lines and fixtures. This is really a big advantage.

In order to achieve 60 jobs an hour, you would need 60 such robot cellseach putting a car together in an hour for example. A big question ishow would you feed the parts to the cells. Note that 60 jobs an hour isa typical number for both body plants and assembly plants but someJapanese run 30. We can take that to be the minimum.

Clearly some sort of very clever material handling would be required inorder to get the parts to these individual drops. The simpliest of all,of course, is an overhead monorail conveyor 914 bringing the parts inwith the robot simply going over just as the human does in atransmission plant and grabbing off the one it wants in place for itsjob. This would be by far the cheapest, but would require numerousconveyor loops which might get in the way of each other since we aretalking about multiple types of parts coming in at once. The loops mighthave to go in a circle around the whole operation with the robot movingradially outward from its Job to find the part it wants, grabbing it offand bringing it back.

A second way is to simply have them in racks 916, bring the racks in (asmany of the parts are racked anyway) and bring them in with trucks toeach of the individual job sites. For example, if each rack contains 16parts, we can do a day's production with a single rack full at any onecell. The true flexibility of this is a distinct advantage.

One of the fascinating aspects of all of this is that probably theoptimum way in which the car was built could be totally laid out on aCAD system. Different cars would use different sequences or differentideas. You could build anything given this approach, just as humanscould in a garage if they were presented with a kit of parts. Note toothat the CAD system could also design the target on the part--specifyingspecial stamped in (cut in, molded in, etc.) targets etc. at certainpoints on the part surface. The CAD system too would not only know thelocation of the targets to other part features, but to all other targetson all other parts used to form the assembly.

I claim:
 1. A method of correcting robot position comprising the stepsof:providing a computer-programmed robot having a work area andprogrammed to execute predetermined working movements relative to anobject to be worked by the robot, providing at least one electro-opticalsensor on said robot, providing at least one reference point fixed in ornear the work area of the robot, sensing said reference point with saidsensor, determining relative position data between said robot and saidreference point, and providing a correction signal to said robot basedon said relative position data for correcting a predetermined workingmovement of said robot.
 2. A method according to claim 1 wherein saidsensor includes a matrix array TV camera.
 3. A method according to claim1 wherein said sensor is a stereo pair of TV cameras.
 4. A methodaccording to claim 1 wherein said reference point is visible when anobject is worked by said robot.
 5. A method according to claim 1 whereinsaid reference point is on a holding device for an object worked by saidrobot.
 6. A method according to claim 1 wherein a plurality of referencepoints are used at different locations in the robot work area.
 7. Amethod according to claim 1 wherein said reference point comprises aplurality of reference points in a group providing additional axesinformation.
 8. A method according to claim 1 wherein said robotincludes a guidance computer which automatically creates a program forsaid robot.
 9. A method according to claim 1 wherein said referencepoint is sensed in at least two axes.
 10. A method according to claim 1wherein said reference point is visible when an object worked by saidrobot is moved away.
 11. A method of correcting robot positioncomprising the steps of:providing a computer-programmed robot having awork area and programmed to execute predetermined working movements,providing at least one electro-optical sensor fixed in or near the workarea of the robot, providing at least one reference point on said robot,sensing said reference point with said sensor, determining relativeposition data between said robot and said reference point, and providinga correction signal to said robot based on said relative position datafor correcting a predetermined working movement of the robot.
 12. Amethod according to claim 11 wherein said sensor includes a matrix arrayTV camera.
 13. A method according to claim 11 wherein said sensorcomprises a stereo pair of TV cameras.
 14. A method according to claim11 wherein said reference point comprises a plurality of referencepoints arranged in a group to provide additional axes information.
 15. Amethod according to claim 11 wherein said robot includes a guidancecomputer which automatically creates a program for said robot.
 16. Amethod according to claim 11 wherein said reference point is sensed inat least two axes.
 17. A method of correcting robot position comprisingthe steps of:providing a computer-programmed robot having a work areaand programmed to execute working movements relative to an object to beworked by the robot providing at least one electro-optical sensorproviding at least one reference point on a holding device for theobject to be worked by the robot, sensing said reference point with saidsensor, determining relative position data between said robot and saidreference point, and providing a correction signal to said robot basedon said relative position data for correcting a predetermined workingmovement of said robot.
 18. A method according to claim 17 wherein saidsensor includes a matrix array TV camera.
 19. A method according claim17 wherein said sensor is a stereo pair of TV cameras.
 20. A methodaccording to claim 17 wherein said at least one reference pointcomprises a plurality of reference points arranged in a group to provideadditional axes information.